Learning Communities Research and Practice 


Volume 1 | Issue 1 


Article 15 


1 - 28-2013 

Learning Community Assessment 101 - Best 
Practices 

Juan Carlos Huerta 

Texas A&M University-Corpus Christi, juan.huerta(Stamucc.edu 

Michele J. Hansen 

Indiana University-Purdue University Indianapolis, mjhansen(8)iupui.edu 


Recommended Citation 

Huerta, J. C., Hansen, M.J. (2013). Learning Community Assessment 101 - Best Practices. Learning Communities Research and 
Practice, 1 (1), Article 15. 

Available at: http://washingtoncenter.evergreen.edu/lcrpjournal/voll/issl/15 


Authors retain copyright of their material under a Creative Commons Non-Commercial Attribution 3.0 License. 





Learning Community Assessment 101 - Best Practices 


Abstract 

Good assessment is part of all good learning communities; and this article provides a useful set of best 
practices for learning community assessment planning: (1) articulating agreed-upon learning community 
program goals; (2) identifying the purpose of assessment (e.g., summative or formative); (3) employing 
qualitative and quantitative assessment methods for assessing the most critical outcomes for administrative 
and instructional team-member decision making processes; (4) employing indirect and direct measures of 
student learning; and (5) ensuring that assessment results are used and that campus decision makers are 
equipped with the information required to create high-quality learning experiences to meet the diverse needs 
of all students. 
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Learning Community Assessment 101 — Best Practices 


Juan Carlos Huerta 
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O ffering learning communities has become a prevailing strategy in 
higher education to ensure that students have enriching academic 
experiences. Learning communities have been advocated as effective 
interventions for enhancing student retention (Tinto, 2003), engagement 
levels (Yancy, Sutton-Haywood, Hermitte, Dawkins, Rainey, & Parker, 
2008; Zhao & Kuh, 2004), student learning and academic success 
(Hegler, 2004; Henscheild, 2004; Huerta, 2009; Kuh, 2008; Stassen, 
2003), opportunities for service learning (Oates & Leavitt, 2003), critical 
thinking (Lardner & Malnarich, 2008), and integrative learning (Lardner 
& Malnarich, 2008, 2008/2009, 2009). 

In this article, we report on the following best practices in the areas 
of planning for learning community assessment: l) articulating agreed- 
upon learning community program goals, 2) identifying the purpose 
of assessment (e.g., summative or formative) and deciding on data, 3) 
employing qualitative and quantitative assessment methods for assessing 
the most critical outcomes for administrative and instructional team 
member decision-making processes, 4) employing indirect and direct 
measures of student learning, and 5) ensuring that assessment results are 
used and that campus decision makers are equipped with the information 
required to create high-quality learning experiences to meet the diverse 
needs of all students. 


Articulating Agreed-Upon 
Learning Community Program Goals and Theory 

Assessing learning community programs requires careful 
conceptualization of the processes and intended outcomes before choosing 
measures and evaluation designs. Assessment scholars have advocated for 
the development of plans with clear purposes closely aligned with valued 
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program goals (e.g., Banta, 2002; Posavac & Carey, 2006; Swing, 2001, 
2004). Typically, programs designed to improve student learning and 
success have multiple outcomes, and a narrow focus on retention may 
obscure an understanding of program processes and effects. Additionally, a 
myopic focus on specific outcomes such as retention rates and grade point 
averages may not be sufficient for making substantive improvements based 
on assessment findings. 

As such, an effective assessment planning approach should begin 
with a clearly articulated learning community program theory and goals to 
help guide the selection of instruments and to increase understanding among 
researchers and administrators regarding what internal program operations 
need to be improved when selected outcome measures suggest that desired 
program “outputs” are not achieved. In other words, the program theory and 
goals need to be clear and well understood by all, otherwise there will be 
disagreements about how and what to assess that will threaten the assessment. 
Ideally, this approach expands the assessment focus to include investigation 
into the underlying assumptions, history, and context associated with a 
particular learning community program. A clear understanding of program 
goals is a critical first step in assessment planning. With agreed-upon goals 
clearly defined, assessment planners will be able to select measures and 
instruments that are sensitive, valid, and reliable. 

Identify the Purpose of Assessment 

A clear understanding of the purpose of the assessment is essential 
because it influences the design and data needs of the assessment. For 
example, classroom teaching techniques or student attitudes might be 
subjects of evaluation. In this case, student surveys would be appropriate. 
If the purpose is to measure student learning outcomes, then a different 
design and data sources are needed. Other purposes may be to assess 
student satisfaction, student and faculty participation, or the impact on 
outcomes such as retention or grades. 

Employ Quantitative and Qualitative Assessment Methods 

Once an understanding of the purpose of the assessment is established, 
attention can then turn to choosing data for evidence. Data fall into two 
categories—quantitative and qualitative. Quantitative and qualitative data 
each have advantages, and, when possible, it is best to use both. 

Quantitative areas assessed may include program impact on academic 
achievement (e.g., grade point averages; numbers of Ds, Fs, Withdrawals, 
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and Incompletes in courses), retention rates, and graduation rates. An 
advantage of quantitative data is that it is obtainable and can be analyzed 
statistically. Employing questionnaires that contain items with Likert-type 
scales may also be useful to enhance understanding of students’ perceptions 
of course benefits, self-reported learning outcomes, and satisfaction levels. 

Qualitative evaluations provide the kinds of in-depth process 
information that allow faculty, staff, and students to better understand 
when and how certain learning community interventions are effective. 
Conducting focus groups and one-on-one interviews can be enormously 
helpful for understanding students’ in-depth perceptions and exploring 
students’ levels of integrative learning. Content analysis of students’ 
open-ended comments on surveys can also provide data that sheds light 
on students’ academic needs that can be addressed in future learning 
community implementations. 

Employ Indirect and Direct Measures of Student Learning 

Indirect measures of student learning are most often derived 
from student surveys by asking students to respond to questions about 
their learning experiences. Direct measures of student learning require 
students to demonstrate their knowledge and skills. They provide 
tangible, visible, and self-explanatory evidence of what students have 
and have not learned as a result of a course, program, or activity (Huerta, 
2009; Suskie, 2004; Palomba & Banta, 1999). Examples of direct student 
learning measures include objective tests, essays, presentations, and 
classroom assignments. The key is to identify assignments, or parts of 
assignments, that allow students to demonstrate mastery of the student 
learning outcome. 

Ensuring That Assessment Results Are Used 

ft is important to make the results of outcomes assessment transparent 
and readily available to all internal and external stakeholders. We have 
discoveredthat fundamental institutional change and continuous improvement 
is more fully realized by sharing critical outcomes and actively discussing the 
processes that create the outcomes. This is a critical part of the assessment 
process. The assessment findings must be analyzed and used for program 
improvement. Schedule meetings with the stakeholders and participants to 
share and reflect on the findings. Take notes at these meetings and use the 
data for making improvements to the program. Successful assessment is a 
cycle, and using the results for program improvement completes the cycle. 
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Steps for Ensuring Successful Assessment Planning: 

Moving the Learning Community Assessment Wheel Forward 

Lardner and Malnarich (2008) have emphasized the importance of 
sharing longitudinal learning community assessment data with key decision 
makers and actively using learning communities ’ assessment data in making 
decisions about the purposes and structures of learning communities, 
identifying curricular trouble spots, improving academic achievement, 
and determining students’ levels of academic preparedness. A “Learning 
Community Assessment Planning Checklist” with links to assessment 
examples is provided in the Appendix. We hope the following are key 
aspects of all campus assessment plans: involving faculty in assessment 
planning; defining and measuring integrative learning; assessing students’ 
learning outcomes by using embedded, authentic assessments; employing 
research designs that identify the program components that have the 
most positive educational outcomes for diverse groups of students; and 
communicating results in a timely manner so they are used when decisions 
are made. 
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Appendix 

Learning Community Assessment Planning Checklist 

□ Identify and actively involve stakeholders (e.g., faculty, staff, advisors, 
program administrators, students) in all planning stages 

□ Identify the purpose of the assessment efforts (e.g., summative or 
formative) 

□ Clearly articulate the agreed-upon program goals, underlying 
assumptions, and theories guiding the program 

□ Consider only instruments and measures that are valid, reliable, 
and aligned with intended student learning outcomes and proposed 
curricula (e.g., assessment and curricula are carefully aligned) 

□ Include direct as well as indirect measures of student learning 

□ Include measures designed to assess cognitive, affective, and social 
outcomes 

□ Consider a combination of quantitative and qualitative methods 

□ Employ research designs with acceptable internal validity (e.g., research 
designs such as pre-post with appropriate comparison groups) 

□ Recruit individuals with the technical expertise required to analyze 
the data and produce useful, actionable reports (e.g., involve the 
institutional research office or other campus offices that employ 
individuals with expertise in quantitative and qualitative data analysis 
and report production) 

□ Make concrete plans for ensuring that the assessment results are used 
and reports are shared with all stakeholders, including key campus 
decision makers 

□ Develop a written plan with stakeholders clearly articulating purposes, 
program goals, definitions, time lines, data collection strategies and 
methods, and reporting strategies 

Please link to the University College Assessment Web site at Indiana 

University-Purdue University Indianapolis to view examples of assessment 

reports, presentations, and measures for various first-year programs, including 

learning communities (http://uc.iupui.edu/staff7assessment/index.asp). 
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